As Kinect®-like devices become more prevalent in the consumer space, they are empowering a brand new generation of user interfaces, ones that are enabled with gesture recognition. Front-facing gesture recognition, with cameras placed in peripheral devices for game consoles or PCs, is very powerful, enabling fun, aerobic gesture controls for games. The inventors of the present invention have recognized that this approach, however, suffers from a number of drawbacks: often times, sensors don't work very well, or even at all, in regions that are close to the screen or are lower in the field-of-view (FOV) of the sensors. Because these are essential areas of interaction for the user, these users are left with the choice of foregoing such interaction, or attempting the interaction at great discomfort. With cameras placed in the bezel of a screen, some regions of desired interaction cannot be viewed by the camera, even if the cameras are provided having a wide field of view. More importantly, for current technologies being utilized in gesture recognition with either time-of-flight (TOF) or structured light, peripherally located objects relative to the FOV may not be easily viewed by the camera, and may have poor reflectivity, vis-à-vis a light source that is associated with camera or image sensor. Many such objects may have slanted surfaces at such locations in the FOV that image distortions, associated with both the lenses and the light source, present a very serious challenge, both, in engineering and usability.
Many are the advantages of gesture recognition, enabling a more natural means of interaction between the user and their surroundings. Gesture recognition today is especially attractive for playing games, by enabling user gestures. This is evidenced by millions of Kinect® users who have taken to such interfaces very naturally. The gesture lexicon that is associated with a gesture recognition system is typically simplistic, involving coarse body gestures and tracking the user's entire body in the FOV, which is impractical for near-touch interaction. In this case, the gesture lexicon itself is monolithic, mostly recycling very similar gestures across multiple games.
While gesture recognition has proven to be very valuable for games, it has been considered less valuable in other settings that require more precise motion tracking, or that require a greater degree of robustness, more integrated interaction with other types of controls, and a significantly more diverse gesture lexicon. For such applications, the idea of using such coarse gestures becomes less appealing. Moreover, user/arm fatigue, among other factors typically becomes more of an issue. It would be desirable to develop another approach for more user-centric gesture recognition, capable of addressing the shortcomings of the current state-of-the-art.
It would therefore be beneficial to present a method and apparatus for overcoming the drawbacks of the prior art.